Remixing Preferences for Western Instrumental Classical Music of Bilateral Cochlear Implant Users.

For people with profound hearing loss, a cochlear implant (CI) is able to provide access to sounds that support speech perception. With current technology, most CI users obtain very good speech understanding in quiet listening environments. However, many CI users still struggle when listening to music. Efforts have been made to preprocess music for CI users and improve their music enjoyment. This work investigates potential modifications of instrumental music to make it more accessible for CI users. For this purpose, we used two datasets with varying complexity and containing individual tracks of instrumental music. The first dataset contained trios and it was newly created and synthesized for this study. The second dataset contained orchestral music with a large number of instruments. Bilateral CI users and normal hearing listeners were asked to remix the multitracks grouped into melody, bass, accompaniment, and percussion. Remixes could be performed in the amplitude, spatial, and spectral domains. Results showed that CI users preferred tracks being panned toward the right side, especially the percussion component. When CI users were grouped into frequent or occasional music listeners, significant differences in remixing preferences in all domains were observed.

A Fused Deep Denoising Sound Coding Strategy for Bilateral Cochlear Implants.

Cochlear implants (CIs) provide a solution for individuals with severe sensorineural hearing loss to regain their hearing abilities. When someone experiences this form of hearing impairment in both ears, they may be equipped with two separate CI devices, which will typically further improve the CI benefits. This spatial hearing is particularly crucial when tackling the challenge of understanding speech in noisy environments, a common issue CI users face. Currently, extensive research is dedicated to developing algorithms that can autonomously filter out undesired background noises from desired speech signals. At present, some research focuses on achieving end-to-end denoising, either as an integral component of the initial CI signal processing or by fully integrating the denoising process into the CI sound coding strategy. This work is presented in the context of bilateral CI (BiCI) systems, where we propose a deep-learning-based bilateral speech enhancement model that shares information between both hearing sides. Specifically, we connect two monaural end-to-end deep denoising sound coding techniques through intermediary latent fusion layers. These layers amalgamate the latent representations generated by these techniques by multiplying them together, resulting in an enhanced ability to reduce noise and improve learning generalization. The objective instrumental results demonstrate that the proposed fused BiCI sound coding strategy achieves higher interaural coherence, superior noise reduction, and enhanced predicted speech intelligibility scores compared to the baseline methods. Furthermore, our speech-in-noise intelligibility results in BiCI users reveal that the deep denoising sound coding strategy can attain scores similar to those achieved in quiet conditions.

A Deep Denoising Sound Coding Strategy for Cochlear Implants.

Cochlear implants (CIs) have proven to be successful at restoring the sensation of hearing in people who suffer from profound sensorineural hearing loss. CI users generally achieve good speech understanding in quiet acoustic conditions. However, their ability to understand speech degrades drastically when background interfering noise is present. To address this problem, current CI systems are delivered with front-end speech enhancement modules that can aid the listener in noisy environments. However, these only perform well under certain noisy conditions, leaving quite some room for improvement in more challenging circumstances. In this work, we propose replacing the CI sound coding strategy with a deep neural network (DNN) that performs end-to-end speech denoising by taking the raw audio as input and providing a denoised electrodogram, i.e., the electrical stimulation patterns applied to the electrodes across time. We specifically introduce a DNN that emulates a common CI sound coding strategy, the advanced combination encoder (ACE). We refer to the proposed algorithm as 'Deep ACE'. Deep ACE is designed not only to accurately code the acoustic signals in the same way that ACE would but also to automatically remove unwanted interfering noises, without sacrificing processing latency. The model was optimized using a CI-specific loss function and evaluated using objective measures as well as listening tests in CI participants. Results show that, based on objective measures, the proposed model achieved higher scores when compared to the baseline algorithms. Also, the proposed deep learning-based sound coding strategy gave eight CI users the highest speech intelligibility scores.

Enhancement of interaural level differences for bilateral cochlear implant users.

Bilateral cochlear implant (BiCI) users do not localize sounds as well as normal hearing (NH) listeners do. NH listeners rely on two binaural cues to localize sounds in the horizontal plane, namely interaural level differences (ILDs) and interaural time differences. BiCI systems, however, convey these cues poorly. In this work, we investigated two methods to improve the coding of ILDs in BiCIs. The first method enhances ILDs by applying an artificial current-versus-angle function to the clinical levels delivered by the basal electrodes of the CI contralateral to the target sound. The second method enhances ILDs by using bilaterally linked N-of-M band selection. Results indicate that the participants were able to discriminate the location of the sound more accurately at narrow azimuths when the ILD enhancement was applied, compared to when they were using natural ILDs. Also, the results show that linking the band selection had a positive effect on left/right discrimination accuracy at larger azimuths for three out of the 10 tested participants, when compared to unlinked band selection. Based on these results, we conclude that ILD enhancement besides linked N-of-M band selection can help some BiCI participants to discriminate sound sources on the frontal horizontal plane.

A subjective and objective evaluation of a codec for the electrical stimulation patterns of cochlear implants.

Wireless transmission of audio from or to signal processors of cochlear implants (CIs) is used to improve speech understanding of CI users. This transmission requires wireless communication to exchange the necessary data. Because they are battery powered devices, energy consumption needs to be kept low in CIs, therefore making bitrate reduction of the audio signals necessary. Additionally, low latency is essential. Previously, a codec for the electrodograms of CIs, called the Electrocodec, was proposed. In this work, a subjective evaluation of the Electrocodec is presented, which investigates the impact of the codec on monaural speech performance. The Electrocodec is evaluated with respect to speech recognition and quality in ten CI users and compared to the Opus audio codec. Opus is a low latency and low bitrate audio codec that best met the CI requirements in terms of bandwidth, bitrate, and latency. Achieving equal speech recognition and quality as Opus, the Electrocodec achieves lower mean bitrates than Opus. Actual rates vary from 24.3 up to 53.5 kbit/s, depending on the codec settings. While Opus has a minimum algorithmic latency of 5 ms, the Electrocodec has an algorithmic latency of 0 ms.

The effect of synchronized linked band selection on speech intelligibility of bilateral cochlear implant users.

Normal-hearing (NH) listeners have the ability to combine the audio input perceived by each ear to extract target information in challenging listening scenarios. Bilateral cochlear implant (BiCI) users, however, do not benefit as much as NH listeners do from a bilateral input. In this study, we investigate the effect that bilaterally synchronized electrical stimulation, bilaterally linked band selection, and ideal binary masks (IdBMs) have on the ability of 10 BiCIs to understand speech in background noise. The performance was assessed through a sentence-based speech intelligibility test, in a scenario where the speech signal was presented from the front and the interfering noise from one side. The linked band selection relies on the most favorable signal-to-noise-ratio (SNR) ear, which will select the bands to be stimulated for both CIs. Speech perception results show that BiCI listeners benefit from adding a second CI to the better-SNR side, obtaining the largest benefit when using bilaterally linked band selection and bilaterally synchronized electrical stimulation. Furthermore, synchronized linked band selection leads to an improvement in speech intelligibility scores when compared to standard clinical BiCI setups. Finally, we observe that by also applying IdBMs, subjects achieve speech intelligibility scores similar to the ones without background noise.

Coding of Electrical Stimulation Patterns for Binaural Sound Coding Strategies for Cochlear Implants.

Binaural sound coding strategies can improve speech intelligibility for cochlear implant (CI) users. These require a signal transmission between two CIs. As power consumption needs to be kept low in CIs, efficient coding or bit-rate reduction of the signals is necessary. In this work, it is proposed to code the electrical signals or excitation patterns (EP) of the CI instead of the audio signals captured by the microphones. For this purpose we designed a differential pulse code modulation based codec with zero algorithmic delay to code the EP of the advanced combination encoder (ACE) sound coding strategy for CIs. Our EP codec was compared to the G.722 64 kbit/s audio codec using the signal-to-noise ratio (SNR) as objective measure of quality. On two audio-sets the mean SNR was 0.5 to 13.9 dB higher when coding the EP with the proposed coding method while achieving a mean bit-rate between 34.1 and 40.3 kbit/s.

Deep learning models to remix music for cochlear implant users.

The severe hearing loss problems that some people suffer can be treated by providing them with a surgically implanted electrical device called cochlear implant (CI). CI users struggle to perceive complex audio signals such as music; however, previous studies show that CI recipients find music more enjoyable when the vocals are enhanced with respect to the background music. In this manuscript source separation (SS) algorithms are used to remix pop songs by applying gain to the lead singing voice. This work uses deep convolutional auto-encoders, a deep recurrent neural network, a multilayer perceptron (MLP), and non-negative matrix factorization to be evaluated objectively and subjectively through two different perceptual experiments which involve normal hearing subjects and CI recipients. The evaluation assesses the relevance of the artifacts introduced by the SS algorithms considering their computation time, as this study aims at proposing one of the algorithms for real-time implementation. Results show that the MLP performs in a robust way throughout the tested data while providing levels of distortions and artifacts which are not perceived by CI users. Thus, an MLP is proposed to be implemented for real-time monaural audio SS to remix music for CI users.